An Efficient Mechanism for Navigating Web Using Mobile Web Crawler

With the fast pace growth of World Wide Web and its dynamic nature coupled with presence of large volume of contents, the web crawlers have become an indispensable part of search engines. The growing use of search engines and their dependency in every day life necessitates that the correct and relevant information is presented to users in response to their search queries. Web crawler plays an important role to meet this objective by playing an important part of a search engine. A mobile web crawler is an automated computer program, which transfers itself to web servers in an attempt to download information and contents. Dynamically changing nature of web requires that mobile web crawlers must be able to intelligently decide about new pages and the changes in already crawled pages. This ability of mobile web crawler allows minimizing the consumption of resources. This paper aims to provide an efficient crawling mechanism for implementing a mobile web crawler, which intelligently decides about page changes to reduce overall load on web resources and helps search engines to increase web crawling speed and expand their reach in indexing.